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SUMMARY 

Team  VaCAS  developed  a  cooperative  unmanned  ground  vehicle  (UGV)  system  for  MAGIC  2010 
competition.  The  system  includes  a  cooperative  estimation  and  control  strategy,  its  software,  UGVs  and  the 
so-called  platform-  and  hardware-in-the-loop  simulator  (PHILS)  which  allows  the  performance  evaluation  of 
multi-UGV  system  in  a  virtual  environment.  Team  VaCAS  also  constructed  outdoor  and  indoor  test  areas  for 
June  site  demonstration.  Out  of  the  developments,  a  cooperative  estimation  and  control  strategy,  which  is 
based  on  the  central  decision  making  and  the  decentralized  Bayesian  estimation  and  control,  enables  the  UGVs 
to  complete  the  mission  of  MAGIC  robustly  while  handling  uncertainties  inherent  and  significant  in  real 
systems  in  natural  environment.  For  June  site  demonstration,  three  UGVs,  each  having  a  different  set  of  drive 
train  and  sensors,  were  developed.  The  PHILS  is  a  multi-computer  multi-monitor  system  and  thus  allows  the 
performance  evaluation  of  various  subsystems.  The  developed  multi-UGV  system  was  first  utilized  for  June 
site  demonstration  using  PHILS  and  its  ability  and  proficiency  were  demonstrated.  The  system  was  then 
applied  to  the  MAGIC  mission  in  a  MAGIC  Final  like  test  area  consisting  of  three  phases,  and  its  ability  to 
complete  the  entire  MAGIC  mission  was  demonstrated.  Further,  the  compatibility  of  virtual  tests  to  real  tests 
were  demonstrated,  and  the  three  real  UGVs  were  cooperated  with  five  simulated  UGVs  in  the  same 
environment. 
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ORIGINAL  PLAN 


INTRODUCTION 

Statement  of  the  Problem 

The  Multi  Autonomous  Ground-robotic  International  Challenge  2010  (MAGIC  2010)  aims  to  improve  the 
effectiveness,  safety,  and  situational  awareness  of  dismounted  ground  forces  when  conducting  urban  zone 
reconnaissance  and  clearance  operations  and  to  improve  the  interaction  between  humans  and  teams  of 
cooperative  unmanned  vehicle  systems  (UVS).  The  system  will  consist  of  unmanned  ground  vehicles  (UGVs) 
of  two  distinct  types  which  will  cooperate  with  each  other  or  act  independently.  The  UGVs  will  also  interact 
with  a  simulated  unmanned  aerial  vehicle  (UAV)  and  minimally  with  a  team  of  operators.  As  a  result  of  the 
complex  makeup  of  the  system,  cooperative  control,  autonomous  task  assignment,  and  the  fusion  of  data  from 
multiple  sources  will  be  emphasized.  During  the  challenge,  the  teams  must  deploy  a  group  of  robust  and 
lightweight  UGVs  to  search  a  mock  urban  environment  for  an  unknown  number  of  static  and  mobile  objects  of 
interest  (001).  The  UGVs  will  be  required  to  detect,  localize,  recognize,  classify,  and  neutralize  001  while 
providing  the  operators  with  a  succinct  yet  thorough  overview  of  the  area  of  operation  (AO).  Finally,  the 
UGVs  must  incorporate  the  uncertainty  of  all  the  states  in  the  system  and  quantify  the  belief  of  target  states  in 
order  to  determine  that  the  AO  has  been  cleared  of  001. 


Conceptual  Solutions  Proposed 

Figure  1  shows  the  conceptual 
solutions  proposed.  The  proposed 
cooperative  UGV  system  will 
consist  of  a  base  station  (BS)  with 
two  operators  and  a  total  of  eight 
UGVs  including  three  disrupter 
UGVs  and  five  sensor  UGVs. 

Due  to  the  minimum  disruptor-to- 
sensor  ratio  within  the 
cooperatives  [Guidelines  10.4], 
two  cooperatives  of  one-disruptor- 
two-sensor  UGVs  will  be  in 
coordination  at  a  time  while  one 
disrupter  UGV  and  one  sensor 
UGV  will  remain  independent  as 
extras.  These  extra  UGVs  will 
join  coordination  when  a  UGV  of 
the  same  type  has  been  killed.  As 
a  result,  two  cooperatives  can  continue  the  mission  with  a  full  complement  of  UGVs  even  if  one  disrupter 
UGV  and  one  sensor  UGV  have  been  disables.  Of  four  sensor  UGVs,  one  sensor  UGV  is  always  a  leader  of 
the  cooperative  UGVs.  As  decentralized  estimation  and  control  becomes  inconsistent  and  non-optimal  after  a 
certain  interval,  the  leader  synchronizes  the  belief  and  regenerates  the  optimal  waypoints. 

Figure  2  summarizes  the  features  of  the  proposed  cooperative  UGV  system.  All  the  UGVs  are  built  on  the 
same  platform  which  is  commercially  available.  The  platform  is  of  differential  steering  type  for  its  high 
manipulability  in  indoor  environments  while  maintaining  a  relatively  high  velocity  profile.  Sensors  mounted 
on  the  UGV  include  those  for  global  positioning,  those  for  localization  and  mapping  and  those  for  dead¬ 
reckoning  such  that  all  the  UGVs  and  objects  of  interest  (OOIs)  can  be  localized  in  both  indoor  and  outdoor 
environments  whilst  a  map  is  created.  Due  to  the  need  for  image  processing  and  nodal  recursive  Bayesian 


Figure  1  Conceptual  solutions  proposed 
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estimation,  graphics  processing  unit  (GPU)  is  implemented  in  addition  to  a  microcontroller  unit  (MCU)  and  a 
central  processing  unit  (CPU). 

Estimations  of  UGVs  and  OOIs  are 
carried  out  using  different  RBE 
techniques  by  accounting  for 

accuracy  and  computational 

efficiency.  Each  UGV  estimates  the 
other  UGVs  and  OOIs  in  a 
decentralized  manner  at  high 
frequency,  whereas  the  leader 

synchronizes  the  estimation  at  low 
frequency.  Control  and  planning  are 
also  performed  at  different 
frequencies.  Driven  at  the  lowest 
frequency  is  strategic  planning  where 
the  leader  creates  discrete  decision 
plans  for  cooperation  and  distributes 
them  to  the  other  coordinating  UGVs. 
High-level  and  low-level  controls  are 
carried  out  at  every  UGV  based  on  various  beliefs  and  attempts  to  create  and  follow  waypoints  respectively 
with  higher-frequency  feedback.  Data  communication  is  also  uniquely  equipped  in  the  proposed  system. 
While  TCP/IP  is  utilized  between  BS  and  UGV,  UGVs  communicate  with  each  other  via  multicast  IP. 


The  UGV  system 

Estimation 

•2  operators 

•  Multi-level  recursive  Bayesian  estimation 

•1  base  station 

•Synchronously  estimated  at  Leader 

•3disruptorUGVs 

•Asynchronously  estimated  at  each  UGV 

•5  sensor  UGVs  (1  Leader) 

Control  and  planning 

UGV 

•Strategic  planning:  rule  driven  and 

•Commercial  platform 

determined  by  Leader 

•  Differential  steering 

•High-/Low-level  control:  belief /waypoint 

•Sensors 

driven  and  determined  at  each  UGV. 

•Global 

•Local 

Data  communication 

•Dead-reckoning 

•TCP/IP  between  BS  and  UGV 

•  MCU  +  CPU  +  GPU 

•  Multicast  IP  between  UGVs 

Figure  2  Summary  of  solutions 


Graphic  Overview  of  Overaii  Systems  Architecture 

The  structure  of  the  rules  for  the  MAGIC  2010  makes  clear  that  the  overall  control  of  the  cooperative  UGV 
system  requires  a  level  of  sophistication  that  is  not  evident  in  existing  commercial,  military  or  academic 
robotic  systems.  The  requirements  of  the  competition  in  fact  go  beyond  the  approaches  that  can  be  found  in 
theoretical  or  numerical  simulation  in  cutting  edge  journals  in  robotics  or  autonomous  systems.  For  example, 
some  theory  has  been  developed  for  cooperative  search  via  collectives  of  autonomous  vehicles,  but  these 
typically  consider  the  control  of  single  vehicles  that  act  cooperatively  with  other  single  vehicles.  References 
[1]  and  [2]  are  examples  of  this  type.  The  MAGIC  2010  requires  controlling  sub-teams  (co-ops)  within  the 
overall  team.  Some  theory  likewise  exists  for  the  networked  allocation  of  vehicles  to  targets,  for  example,  [3], 
but  again,  the  necessity  that  co-ops  be  used  in  the  MAGIC  2010  goes  beyond  the  working  assumptions  in  these 
approaches.  More  importantly,  the  cooperative  search  task  must  be  carried  out  in  consideration  of  engagement 
with  active  and  mobile  enemy  teams,  which  renders  existing  approaches  inapplicable. 


Figure  3  shows  the  graphic 
overview  of  the  overall 
systems  architecture 

proposed  in  the  project. 
Original  contributions  of 
this  architecture  are  three 
multi-level  formulations; 
control,  estimation  and 
belief  representation. 
Owing  to  the  multi-level 
control  and  estimation 
formulations,  the 

architecture  introduces 
three  feedback  loops, 
aimed  at  low-level  control. 


Figure  3  Overall  systems  architecture 
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high-level  control  and  online  strategic  planning.  Out  of  the  loops,  driven  at  the  highest  frequency  is  low-level 
control,  which  is  achieved  by  recursively  updating  beliefs  and  resultantly  following  the  specified  waypoints. 
The  beliefs  updated  at  each  UGV  include  those  on  all  the  UGVs  including  itself  and  those  on  all  the  known 
OOIs.  If  the  beliefs  have  seen  a  significant  change  in  themselves,  the  specified  waypoints  are  no  longer 
effective.  High-level  control  creates  a  new  set  of  waypoints  by  predicting  the  beliefs  in  multiple  steps  as 
model-predictive  control  [4,5]  does  and  extracting  and  optimizing  a  quantity  such  as  information  entropy  and 
information  gain  from  the  beliefs.  The  least  frequent  loop  allows  a  new  strategic  plan  to  be  created.  This  is 
essential  when  OOIs  or  other  environmental  factors  could  force  UGVs  to  re-plan  their  trajectories  based  on  the 
rule-based  reasoning.  Lastly,  the  multi-level  belief  representation  describes  belief  of  states  within  the  system 
through  three  methods.  Belief  for  001  is  constructed  in  terms  of  an  elemental  non-Gaussian  distribution  while 
belief  of  each  UGV  is  in  terms  of  a  Gaussian  distribution  (a  mean  and  a  covariance)  via  the  extended  element- 
based  method  (EEM),  an  efficient  non-Gaussian  recursive  Bayesian  estimator  developed  by  the  investigators 
[6,7],  and  the  extended  Kalman  Filter  (EKE)  [8]  respectively.  Additionally  a  map  is  created  via  occupancy 
element  map  (OEM)  [7],  which  was  also  developed  by  the  authors.  It  is  to  be  noted  here  that  the  use  of  the 
EEM  is  due  to  the  fact  that  the  beliefs  of  non-cooperating  OOIs  become  heavily  non-Gaussian  through  search 
whilst  the  use  of  the  EKE  for  UGVs  is  because  the  states  of  UGVs  are  well  estimated  by  Gaussian  distributions 
through  communication. 

An  essential  ingredient  in  this  architecture  is  a  multi-level  structure  that  makes  provision  for  the  co-ops  in  the 
competition.  In  addition  to  providing  an  explicit  representation  of  the  co-ops  within  the  overall  team,  the 
multi-level  structure  likewise  serves  an  important  role  in  terms  of  complexity.  It  is  well  known  that  the 
determination  of  policies  that  map  beliefs  to  actions  can  be  prohibitively  expensive  in  all  but  the  most 
rudimentary  models  due  to  either  the  dimension  of  the  state  space  or  the  exponential  growth  of  histories  in 
time.  Casting  the  formulation  in  terms  of  a  multi-level  structure  will  reduce  computational  cost,  particularly  in 
the  determination  of  overall  strategies  for  the  teams.  Team  VaCAS  has  a  long  history  in  fielding  some  of  the 
best  robotic  systems  in  the  most  competitive  forums  in  the  world,  and  moreover,  has  a  unique  theoretical 
expertise  with  the  tools  and  techniques  necessary  to  field  a  winning  team.  The  approach  taken  in  this  proposal 
builds  on  this  expertise  and  synthesizes  techniques  from  game  theory,  partially  observable  Markov  decision 
processes  (POMDP)  and  hierarchical  state  space  models  to  derive  a  theoretical  framework  to  treat  the  strategic 
formulation  of  the  MAGIC  2010  competition. 


Work  Breakdown  and  Milestones 

Identifying  the  conceptual  solutions  and  overall  systems  architecture.  Team  VaCAS  has  been  broken  down 

into  six  groups  with  milestones  as  follows: 

1.  Strategic  planning  group:  Develops  and  implements  a  rule-based  decision  maker  to  cope  with  the  rules 
specified  by  the  organizers  with  effective  strategies. 

2.  High-level  control  group:  Develops  and  implements  a  high-level  control  strategy  which  ultimately 
creates  waypoints  in  a  finite  horizon  at  a  low  frequency. 

3.  Low-level  control  group:  Develops  and  implements  a  low-level  control  strategy  which  ultimately  allows 
the  UGVs  to  follow  the  specified  waypoints  while  avoiding  collision  at  a  high  frequency. 

4.  System  integration  group:  Designs  or  selects  all  the  mechatronic  components  and  integrates  all  to 
complete  the  proposed  multi-UGV  system. 

5.  UGV  group:  Designs  and  develops  the  UGV  platform  and  implements  all  the  mechatronic  components 
onto  the  platform. 

6.  Perception  group:  Develops  an  image  processing  and  analysis  tool  specifically  for  possible  OOIs 
introduced  in  the  MAGIC  2010  but  also  for  standard  landmarks  and  structures. 
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GROUND  VEHICLE  COMPONENT  &  SYSTEMS 


Figure  4  shows  the  ground  vehicle  components.  The  major  components  of  each  UGV  are  (1)  sensors,  (2) 
actuators,  (3)  E-stop,  (4)  processors,  (5)  wireless  modules  and  (6)  battery.  Sensors  include  (a)  GPS  and 
compass  for  absolute  localization,  (b)  laser  range  finder  and  two  cameras  for  relative  localization  and  mapping 
and  (c)  inertia  measurement  unit  (IMU)  in  conjunction  with  encoders  for  dead-reckoning.  Two  cameras, 
mounted  on  the  right  and  left  edges  of  the  vehicle  front,  are  pan/tilt  cameras,  so  that  they  can  be  utilized  not 
only  a  stereo  vision  range/bearing  sensor  but  also  as  two  independent  bearing  sensors.  Two  wireless  modules 
are  implemented  to  relay  data  between  a  UGV  and  the  base  station  or  between  two  vehicles. 


Most  unique  in  the  developed  system  are  a 
GPU  and  different  protocols.  Image 
processing  and  non-Gaussian  RBE,  required 
for  estimating  the  beliefs  of  OOIs,  both  see 
node-by-node  calculations.  The  nVidia 
GPU,  together  with  CUBA  library  for 
general-purpose  computing  on  graphics 
processing  units  (GPGPU)  accelerates,  not 
only  image  processing,  but  also  RBE  such 
that  real-time  RBE  is  possible.  This  is 
indicated  by  successful  demonstrations  by 
the  investigators  [9,10]  using  GPU  which 
produced  computational  speed 

improvements  of  more  than  two  orders. 
The  use  of  two  different  protocols,  TCP/IP 
and  multicast  IP,  is  based  on  additional 
research  by  the  investigators  [11]. 
Observing  that  cooperative  autonomous 
vehicles  require  frequent  communication 
even  at  the  expense  of  reliability  whilst  the 
communication  between  the  base  station  and  an  autonomous  vehicle  could  be  less  frequent  but  should  be 
reliable,  multicast  IP  and  TCP/IP  are  utilized  for  UGV-UGV  and  BS-UGV  communications,  respectively. 


Figure  4  Ground  vehicle  components 


The  chassis  of  each  UGV  is  based  on  a 
Superdroidrobots  Heavy  Duty  4WD  RC  All  Terrain 
Robot  Kit,  shown  in  Figure  5.  Since  the  UGVs  are  of 
a  4WD  differential  steering  design,  they  are  able  to 
not  only  travel  across  a  wide  range  of  terrains, 
including  gravel  paths,  but  also  navigate  in  confined 
areas  with  ease  due  to  their  zero  tuning  radii.  The 
maximum  speed  is  6.4  km/h  and  the  maximum 
payload  for  onboard  equipments  is  10  kg,  while  the 
ground  clearance  and  battery  life  will  be  more  than 
0.1  m  and  3.5  hours  respectively  after  slight 
modifications.  The  total  weight  of  each  UGV  with  all 
equipments  is  less  than  20  kg,  while  the  width  and 
height  are  less  than  0.7  m  and  1  m,  respectively.  The 
mechanical  properties  of  the  UGVs  well  suit  the 
challenges  in  the  MAGIC  2010. 


Figure  5  Mechanical  base  of  a  UGV 


In  order  to  demonstrate  the  ability  of  the  investigators  in  developing  such  UGVs,  Figure  6  shows  some  of  the 
autonomous  UGVs  developed  or  integrated  by  the  investigators.  Shown  in  the  upper  left  are  the  award¬ 
winning  UGVs  (Best  UGV  Performance  Award  at  MAV08),  which  have  the  full  capability  for  cooperation 
within  UGVs  and  with  rotary-wing  micro  aerial  vehicles  via  BS.  The  upper  right  figure  shows  the  vehicle 
developed  for  DARPA  Urban  Challenge,  with  which  Virginia  Tech  team  received  the  third  prize.  The  pictures 
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in  the  middle  left  show  a  small  scale  UGV  newly  developed  with  full  autonomous  capabilities;  whilst  the 
bottom  pictures  show  full  scale  autonomous  UGVs  that  the  investigators  integrated  for  cooperative  control 
demonstration.  Finally,  shown  in  the  right  end  is  the  article  of  the  ground  vehicle  that  the  investigators 
developed,  which  was  on  Washington  Post  on  August  1,  2009, 


Figure  6  UGVs  developed/integrated  by  the  investigators 

UVS  AUTONOMY  &  COORDINATION  STRATEGY  (BY  TASK) 

Multi-level  Estimation 

Multi-level  estimation  overviewed  in  Introduction  is  developed  based  on  the  previous  pioneering  theoretical 
work  of  the  investigators  in  (1)  Extended  RBE  (ERBE)  [12,13],  (2)  Partially  Observable  Markov  Decision 
Process  (POMDP)  with  negative  likelihood  [14-21],  (3)  Element-based  method  with  GPU  [9]  and  (4)  Multi¬ 
level  EEM/EKF  RBE  [13].  In  order  to  explain  the  significance  and  originality  of  the  ERBE,  let  me  describe 
the  framework  of  the  standard  RBE,  which  consists  of  correction  and  prediction  as 

\br=fc[bmm^)  (1) 

where  ,  /f  and  mf  are  the  belief,  the  empirical  knowledge  and  the  received  messages  of  UGV  i  at  time  k 
respectively,  is  the  Markovian  motion  model,  and  and  b^^^  are  the  corrected  belief  at  time  k  and 

the  predicted  belief  at  time  ^  + 1 .  This  framework  suffices  if  the  belief  is  of  a  static  001  or  if  the  RBE 
technique  is  the  one  which  does  not  maintain  the  belief  space  configuration  such  as  EKF  and  particle  filters. 
However,  if  the  001  is  mobile  and  is  not  detectable  (Some  OOIs  in  MAGIC  2010  are  indeed  mobile  and  not 
detectable),  its  belief  must  be  maintained  with  its  space  configuration  dynamically.  The  ERBE  extends  the 
standard  RBE  by  additionally  introducing  the  reduction  and  expansion  of  the  belief  space  as 

\  (2) 

where  and  B^^^  are  the  original,  reduced  and  expanded  belief  space. 

In  order  to  maintain  and  update  the  belief  effectively  within  the  framework,  the  POMDP  developed  by  the 
investigators  defines  the  probability  of  detection  and  constructs  a  negative  observation  likelihood  if  the  001  is 
not  within  the  field-of-view  and  thus  not  detected.  Let  the  unknown  state  be  given  by  xf  .  The  observation 
likelihood  is  given  by 
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Detected 


(3) 


1^1  -  yx\  I  /f  j  Not  detected 

where  is  the  probability  of  detection  and  p  is  a  likelihood  defined  when  the  001  has  been  detected. 
Despite  its  effective  estimation,  it  is  however  to  be  remarked  here  that  the  developed  POMDP  will  make  the 
updated  belief  severely  non-Gaussian.  The  RBE  technique,  therefore,  must  have  the  ability  to  update  a 
severely  non-Gaussian  belief  with  its  space  configuration  dynamically. 

The  element-based  method  developed  by  the  authors  adopts  the  concept  of  the  grid-based  method,  which  could 
satisfy  the  requirements  by  representing  the  belief  space  in  terms  of  a  set  of  grid  cells,  but  reformulates  the  grid 
cells  by  elements  with  shape  functions  such  that  the  belief  space  is  represented  by  fewer  parameters  and  thus  is 
computed  less  expensively.  The  past  experimental  investigations  of  the  authors  show  the  450%  computational 
improvement  by  the  element-based  method  over  the  grid-based  method  and  further  the  97%  parallel  efficiency 
by  its  implementation  into  GPU  (Speedup  of  31.3  for  32  processors;  Figure  7).  As  a  result,  RBE  on  a  belief 
space  equivalent  to  1,000,000  grid  cells  was  performed  within  0.01  second  per  iteration,  demonstrating  its  real¬ 
time  capability  for  RBE  over  a  large  belief  space. 


Figure  7  Computation  time  and  Figure  8  Multi-UGV  cooperative  control  experiment 

speedup 

Lastly,  ERBE  using  EEM  (the  element-based  method  extended  to  dynamically  update  the  belief  space)  and 
EKE  is  another  technique  developed  by  the  investigators  to  reduce  the  computational  load.  Since  UGVs  can 
receive  the  observed  states  of  the  other  UGVs  through  communication,  the  states  of  UGVs  can  be  well 
estimated  by  EKE,  which  requires  only  the  update  of  the  mean  and  the  covariance  rather  than  the  belief 
distribution  on  the  entire  belief  space  and  thus  reduces  the  computational  load. 

The  series  of  technical  developments  will  maintain  beliefs  most  reliably  and  effectively  while  lightening  the 
computational  load  and  allowing  real-time  capability  for  estimation  and  usability  even  for  low-level  control. 
In  addition,  asynchronous  estimation  on  each  UGV  in  a  decentralized  manner  with  only  occasional 
synchronization  by  the  leader  makes  the  ERBE  highly  scalable  without  delay  by  the  increase  of  the  number  of 
UGVs.  To  date,  the  practical  applicability  of  the  developed  multi-level  estimation  was  experimentally 
investigated  as  shown  in  Figure  8  using  up  to  4  UGVs.  The  experimental  results  have  well  demonstrated  the 
real-time  capability  even  in  a  real  outdoor  environment. 


Multi-level  Control 

Low-level  Control 

Although  low-level  control  is  most  commonly  achieved  deterministically,  the  proposed  architecture  designs 
low-level  control  with  the  beliefs  updated  recursively  as  input.  This  is  due  to  the  following  reasons: 

•  Generality:  The  deterministic  control  can  be  treated  as  a  special  case  of  the  belief-driven  stochastic  control. 
Since  high-level  control  is  based  on  RBE,  the  formulation  of  low-level  control  based  on  RBE  allows  both  to 
be  described  in  the  same  framework. 
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•  Richness:  In  case  of  communication  loss,  obstacle  detection  and  other  unexpected  situations,  following  the 
specified  waypoints  no  longer  a  solution.  Having  various  uncertainties,  belief-driven  control  that  can  utilize 
such  uncertainties  and  determine  its  control  from  richer  information  may  handle  the  situation  better. 

To  first  show  the  ability  in  waypoint  tracking,  let  the  targeted  waypoint  at  time  ^  +  1  and  the  mean  of  the 
estimated  position  of  the  UGV  i  at  time  k  be  and  respectively.  Having  the  position  of  the  UGV  at 

time  k  +  l  predicted  as  =  fp^  ) ,  the  low-level  control  finds  the  control  action  wf  as 

iif  =  arg  max  ||xf (V* ,  wf  )f  (4) 

Note  that  the  low-level  control  is  belief-driven  because  the  mean  is  a  property  of  belief  represented  by  a 
Gaussian  distribution.  Since  the  low-level  control  is  performed  at  high  frequency,  it  is  adequate  to  assume  that 
the  UGV  is  well  approximated  by  linear  equations.  As  a  consequence.  Equation  (4)  will  be  reformulated  as  a 
Linear-Quadratic-Gaussian  (LQG)  problem  and  solved  analytically  for  fast  computation. 

When  pre-specified  waypoint  control  is  no  longer  a  solution  due  to  communication  loss,  obstacle  direction  or 
other  unexpected  situations,  control  should  be  based  on  the  beliefs  of  others  such  as  the  detected  obstacle, 
UGVs  or  OOIs  at  least  until  high-level  controller  can  find  a  new  set  of  waypoints.  Since  the  beliefs  could  be 
Gaussian  or  non-Gaussian,  the  control  solution  wf  could  be  generically  expressed  with  beliefs  and  belief 
spaces  as 

wf  =  arg  max  J  ( wf  |  )  (5) 

This  problem  requires  optimization,  which  consumes  more  computation  than  the  derivation  of  analytical 
solution  due  to  the  repetition  of  predictions.  However,  due  to  the  determination  of  decentralized  control 
actions  for  only  one-step  look-ahead,  only  two  parameters  (two  motor  speeds)  need  to  be  optimized  and  the 
computation  time  does  not  increase  significantly.  The  past  experiments  by  the  investigators  show  that  the 
belief-driven  low-level  control  can  be  achieved  within  0.1  second  on  a  belief  space  equivalent  to  1,000,000 
grid  cells,  indicating  its  ability  for  real-time  control  [9,10]. 


High-level  Control 

The  proposed  system  implements  high-level  control  to  engage  UGVs  with  OOIs  assigned  through  strategic 
planning.  Due  to  the  development  of  the  ERBE  framework,  the  high-level  control,  aimed  at  determining 
control  actions  on  a  finite  time  horizon  up  to  time  /:  +  - 1 ,  is  given  by  an  extension  of  the  low-level  control: 

=argmax/(Mf""‘-‘  (5) 

which  is  solved  by  performing  prediction  times  where  is  the  number  of  optimization  loops.  Since  the 
number  of  control  parameters  is  2n^  and  thus  makes  large,  the  computation  time  becomes  significant.  In 
order  to  achieve  high-level  control  within  the  order  of  a  few  seconds,  the  technique  implemented  keeps  low 
by  differing  the  control  time  interval  from  the  simulation  time  interval  and  specifying  the  control  time  interval 
by  a  multiplication  of  the  simulation  time  interval.  This  reduces  by  the  multiplication  factor  although  the 
optimality  of  the  solution  is  weakened.  Similarly  to  the  low-level  control,  the  feasibility  of  this  approach  has 
already  been  demonstrated  experimentally  [22]. 

Since  the  primary  aim  of  the  high-level  control  is  to  engage  UGVs  with  OOIs,  the  tasks  to  be  defined  are 
search  (when  OOIs  are  not  detected)  and  tracking  (when  OOIs  are  detected).  The  investigators  are  the 
pioneers  of  the  search-and-tracking,  writing  the  first  paper  on  this  topic  [18].  In  their  approach,  search-and- 
tracking  is  not  manipulated  by  control  actions.  It  is  determined  by  the  belief  update  in  measurement  as 
Equation  (3)  described.  Since  the  negative  likelihood  swipes  the  observed  region  of  the  belief  space,  search 
can  be  performed  similarly  to  tracking  by  simply  moving  towards  a  high  belief  region  without  re-visiting  the 
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observed  regions.  If  the  belief  of  the  001  of  concern  is  given  by  |  ,  the  control  solution  can 

be  uniformly  computed  by  minimizing  the  information  entropy  (or  maximizing  the  information  gain): 

^  argmin-| \  |  •  (6) 


(a)  Information  entropy 


(b)k=l 


(c)  k=20 


Figure  9  Cooperative  search  and  tracking 

Figure  9  shows  the  result  of  cooperative  search  and  tracking  that  the  investigators  reported  previously  where 
four  UGVs  search  for  and  track  three  targets.  It  is  shown  that  the  proposed  technique  could  successfully 
search  for  and  track  targets  by  minimizing  the  information  entropy.  The  graph  also  shows  that  the  proposed 
technique  could  discover  targets  tracked  but  lost  early  since  the  belief  is  maintained  unlike  conventional 
deterministic  techniques. 


Strategic  Planning 

Strategic  planning  against  rules  regulated  for  MAGIC  2010  involves  the  study  of  knowledge  representation 
and  reasoning.  While  learning  can  automate  the  process  and  offer  more  academic  contents,  the  strategy 
adopted  is  the  manual  construction  of  an  expert  system  that  constructs  database  and  knowledgebase  and 
associates  them  via  a  number  of  if-then  rules.  The  main  reason  for  the  selection  of  this  approach  is  the 
requirement  of  constructing  only  a  limited  number  of  rules  for  the  MAGIC  2010.  Due  to  the  lack  of  time  that 
was  available  after  the  Participants  Conference  in  August  2009,  database,  knowledgebase  and  rules  to  apply 
for  MAGIC  2010  scenario  and  rules  have  not  been  developed,  yet.  Members  of  the  strategic  planning  group 
have  been  however  identified.  There  is  enough  staff  and  time  available  to  construct  and  implement  all  before 
the  MAGIC  2010. 


Table  1  Sensors  and  processors 


Sensors/Processors 

Model 

2  cameras  with  PTU 
Laser  range  finder 

GPS 

Compass  with  IMU 
Encoders 
Microcontroller 
CPU/GPU 

Toshiba  IK-WB15A 

Hokuyo  UTM-30LX 

Garmin  GPS  18x  5Hz 

Ocean  Server  OS500-S 
Platform  built-in 

AVR  Atmega  8  SPA 
nVidia  ION  (Intel  Atom 
CPU/nVidia  GeForce9400) 

Sensors,  Processing  &  Mapping  for 
UGVs 

Table  1  shows  the  sensors  and  processors  selected  for 
the  UGVs.  They  may  be  replaced  by  other  systems  if 
systems  with  better  performance  in  cost  or  technical 
specifications  are  found.  Most  importantly,  global 
positioning,  localization/mapping  and  dead-reckoning 
can  be  achieved  by  the  sensors;  GPS  and  compass  for 
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global  positioning,  2  cameras  and  laser  range  finder  for  localization/mapping  and  IMU  and  encoders  for  dead¬ 
reckoning.  The  microcontroller  and  the  CPU/GPU  set  were  selected  with  the  extensive  experiences  of  the 
investigators  in  using  these  systems  for  real-time  ERBE  and  image  processing. 

So  as  to  develop  a  map  within  the  ERBE  framework,  the  technique  proposed  in  this  project  is  the  occupancy 
element  mapping,  which  is  also  the  original  contribution  of  the  investigators.  This  technique,  similarly  to  the 
occupancy  grid  mapping  [23],  represents  the  probability  that  each  cell  is  occupied,  but  the  probability  is 
variably  represented  using  shape  functions  similarly  to  the  element-based  method,  requiring  less  computational 
efforts  to  achieve  the  same  accuracy.  Because  of  the  usage  of  the  same  data  structure  and  method,  the 
occupancy  element  method  can  also  take  significant  advantages  from  its  implementation  onto  GPU.  During 
the  map  creation. 


OPERATIONS  IN  GPS-DENIED  ENVIRONMENTS 

GPS  data  from  satellites  may  become  completely  unavailable  if  UGVs  are  in  indoor  environments.  While  the 
global  sensors  including  a  GPS  and  a  compass  and  the  relative  sensors  including  cameras  and  a  laser  range 
finder  are  used  for  global  positioning  and  mapping  in  outdoor  environments,  the  UGVs  additionally  use 
encoders  and  an  IMU  to  compensate  for  the  unavailability  of  GPS.  Much  work  has  already  been  reported  for 
indoor  localization  and  mapping  in  the  name  of  indoor  Simultaneous  Localization  and  Mapping  (SLAM)  [24]. 
Similar  to  such  work,  the  technique  to  be  used  in  the  project  will  also  (1)  use  the  last  reliably  available  data  as 
the  reference  global  position,  (2)  estimate  the  position  after  movement  using  encoders  and  IMU,  and  (3) 
correct  the  position  by  pattern-matching  the  observable  static  objects  using  cameras  and  a  laser  range  finder. 


PROCESSING  AND  FUSION  OF  PROVIDED  METADATA  (FROM  UAV) 

The  vector  map  of  the  challenge  area  and  locations  of  001  detected  by  the  UAV  are  extracted  from  the 
provided  metadata.  To  extract  the  vector  map,  image  processing  and  analysis  techniques  are  used  to  classify 
and  locate  objects  in  the  provided  overhead  EO  image.  The  locations  of  the  classified  objects  are  stored  in  the 
vector  map  that  is  examined  by  the  operators.  Due  to  the  limited  resolution  of  the  EO  image  and  detected  001 
locations  in  the  metadata,  the  uncertainties  of  object  and  001  locations  are  high  and  thus  also  included  in  the 
vector  map.  The  vector  map  gives  a  global  sensor  reading  that  is  fused  with  more  accurate  local  sensor 
information  collected  by  UGVs’  explorations  in  real  time  to  yield  better  belief  of  001  locations  and  map. 


HUMAN-MACHINE  INTERFACE  (HIM) 


Due  to  the  interaction  of  two 
operators  with  eight  UGVs,  it 
is  essential  that  the  human- 
machine  interface  (HIM) 
effectively  allow  the  operators 


In  all  the  stages,  most  decisions  are  made  autonomously,  but  the  operators  are  responsible  for  assessing  the 

decisions  through  the  HMI  and  overriding  them  if  necessary.  For  example,  if  any  UGVs  are  lost  due  to  sniper’s 

action  or  undetected  001,  the  base  station  proposes  re-tasking  and  re-forming  the  teams  based  on  the  belief  of 

001  and  status  of  UGVs.  The  ^ - - 

- OpenUMI  Vision 


operators  then  examine  the 
proposed  solutions  and  decide 
to  accept  or  to  modify  the 
solutions. 


to  interact  with  the  UGVs  and 
monitor  their  performance. 


Figure  10  OpenUMI 
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The  Virginia  Tech  team  will  use  a  high-performance  HMI  which  will  be  extensively  developed  by  their 
industrial  partner,  Defense  Technologies,  Inc.,  who  is  the  developer  of  the  Open  Unmanned  Mission  Interface 
(OpenUMI).  Figure  10  shows  a  graphical  user  interface  of  the  OpenUMI  as  well  as  the  graphical  summary  of 
its  features.  The  OpenUMI  provides  an  interoperability  on  a  common  control  operator  interface  for  multiple, 
heterogeneous  unmanned  vehicle  using  the  STANAG  4586  and  JAUS  standards.  Open  UMI  also  provides 
cross  platform  support  and  is  tested  in  the  Windows  and  Linux  environment,  so  that  it  will  be  immediately 
usable  for  UGVs  of  the  Virginia  Tech  team.  Traditional  control  stations  suffer  from  limitations  of  only  being 
able  to  control  a  single  vehicle  or  a  single  type  of  vehicle.  The  OpenUMI  has  been  developed  to  control  any 
unmanned  system  concurrently,  including  UGVs,  UAVs  and  SUVs. 


OPERATIONAL  APPROACH/MISSIONS  OPERATIONS  STRATEGY 

The  overarching  mission  operations  strategy  is  to  finish  the  mission  as  soon  as  possible  while  minimizing 
damage  to  UGVs  and  non-combatants.  Accordingly  the  behavior  of  the  UGVs  will  progress  from  lowest  risk 
to  highest  risk  so  that  increased  situational  awareness  that  develops  throughout  each  phase  will  mitigate  the 
risk  of  behaviors  such  as  searching  in  buildings  or  confined  spaces.  The  strategy  therefore  is  divided  into  three 
stages  within  each  phase.  In  stage  1,  two  sensor  UGVs  with  video  cameras  equipped  with  telephoto  lenses 
search  for  and  identify  hostile  001  and  non-combatants  and  neutralize  hostile  001  from  long  distances.  Long¬ 
distance  searching  is  extremely  effective  in  large  areas  and  protects  the  UGVs  from  mobile  hostile  OOTs 
attack.  These  UGVs  are  protected  from  static  001  by  the  other  two  sensor  and  two  disruptor  UGVs  sweeping 
paths.  The  goal  in  this  phase  is  to  quickly  clear  the  lowest  risk  regions  thus  creating  a  safe  avenue  of  swift 
retreat  in  the  event  that  UGVs  need  to  flee  a  mobile  001  in  subsequent  stages.  In  stage  2,  after  successfully 
locating  and  neutralizing  most  hostile  001  in  open  and  easy  to  clear  regions,  two  teams,  each  with  one 
disruptor  and  two  sensor  UGVs  with  wide-angle  zoom  video  cameras,  are  sent  to  search  for  remaining  001 
and  a  build  high-resolution  map  in  outdoor 
unexplored  areas.  Here  the  teams  assume  slightly 
more  risk,  searching  confined  spaces,  areas  obscured 
from  aerial  images  such  as  beneath  trees,  and  blind 
spots  around  building  comers.  Stage  2  ends  when 
the  team  and  operators  are  confident  that  the  exterior 
areas  have  been  cleared  of  001.  Finally  in  stage  3, 
the  two  teams  continue  their  exploration  and 
neutralization  in  open  buildings.  In  all  stages 
confidence  that  an  area  had  been  cleared  will  be 
created  by  both  a  low  probability  of  an  undetected 
001  and  low  information  entropy  associated  with 
the  location  of  objects  within  the  area.  The  scalar 
values  of  probability  of  detection  and  information 
entropy  that  will  serve  as  thresholds  to 
systematically  transition  between  stages.  These 
thresholds  will  be  tuned  through  testing  the  final 
vehicles  in  real  buildings  at  Virginia  Tech. 


RISK  REDUCTION  STRATEGY 
EMI/RFI  &  Electrical 

Reduction  of  electro-magnetic  interference  (EMI)  and  radio  frequency  interference  (RFI)  is  achieved  through 
two  solutions,  which  are  shielding  and  proper  placement  of  all  EMI/RFI  sources  and  sensitive  components. 
Figure  11  illustrates  a  cross-sectional  side  view  of  a  simplified  layout  of  the  developing  UGV,  with  color 
highlighted  EMI/RFI  sources  and  sensitive  components.  All  the  sensitive  components  are  shielded  by 


Figure  11  Simplified  layout  of  developing  UGV 
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electrically  grounded  aluminum  cases.  On  the  other  hand,  the  compass  with  IMU,  which  is  extremely  sensitive 
to  EMI/RFI,  is  located  at  a  position  not  in  the  radiation  pattern  of  the  wireless  module  antennas  and  far  away 
from  the  other  major  EMI/RFI  sources.  To  make  internal  communications  more  robust  to  EMI/RFI,  all  signal 
transmissions  between  electrical  devices  are  in  digital  forms. 

For  fire  prevention,  a  few  safety  measures  are  taken  to  reduce  the  risks  of  short  circuit  and  overdrawing  current 
from  the  battery.  In  order  to  prevent  short  circuit  in  raining  conditions,  all  the  electrical  components  are 
located  in  water-proof  cases.  If  a  short  circuit  occurs  in  some  devices,  fuses  will  cut  the  power  to  the  devices 
to  avoid  overheating.  On  the  other  hand,  the  battery  voltage  is  measured  by  the  MCU  and  monitored  by  the 
operators  through  the  HMI  in  the  BS.  The  operators  are  responsible  for  appropriate  high-level  path  planning  to 
prevent  running  out  of  the  battery  before  entering  designated  servicing  zone  (DSZ),  while  the  MCU  will 
trigger  a  termination  mechanism  to  avoid  flattening  the  battery  when  the  battery  voltage  is  too  low. 


Vibration  &  Physicai 

Vibration  isolation  in  each  UGV  is  achieved  by  the  uses  of  large  and  deformable  wheels  and  vibration 
absorption  materials  that  support  critical  components  sensitive  to  vibration,  such  as  the  laptop.  The  front 
wheels  and  rear  wheels,  as  the  front-most  and  rear-most  parts  as  illustrated  in  Figure  11,  also  act  as  soft 
bumpers  to  sustain  impacts  caused  by  collusion.  To  reduce  the  damage  to  surrounding  caused  by  collision, 
sharp  edges  and  corneas  are  avoided  or  covered  by  soft  materials.  To  diminish  the  risk  of  rollover,  most  heavy 
components,  including  battery,  motors  and  laptop,  are  placed  as  low  as  possible  to  minimize  the  height  of  the 
center  of  mass.  In  addition,  the  maximum  speed  and  tuning  rate  are  limited  by  onboard  software. 


Modeling  &  Simulation 

Figure  12  shows  the  modeling  and  simulation 
systems  that  have  been  developed  by  the 
investigators  and  will  be  used  for  MAGIC  2010. 

In  addition  to  the  traditional  hardware-in-the- 
loop  simulation,  the  cooperative  control  will 
require  the  simulation  of  real-time  performance 
of  sensor  platforms  under  cooperation  due  to  the 
high  complexity  of  cooperation.  This  so-called 
Platform-In-the-Loop  Simulator  (PIES)  has  been 
uniquely  developed  to  satisfy  such  need  while 
also  allowing  real  platforms  to  participate  in  the 
cooperation.  As  shown  in  Figure  12(a),  the  PIES 
connects  computers  through  the  server-client 
system.  Since  this  server-client  system  allows 
different  visualization  tools  to  be  communicated, 
there  is  no  need  for  developing  a  new  visualizer 
for  cooperative  control.  Figure  12(b)  shows  a 
PIES  that  is  displaying  cockpit  views  from  eight 
cooperative  UAVs  (eight  monitors  above), 

locations  of  the  UAVs  on  Google  Map  (bottom- 
left)  and  Google  Earth  (bottom-middle)  and  the 
belief  of  a  target  (bottom-right).  Shown  in 
Figure  12(c)  is  a  mobile  PIES,  which  has  been  developed  to  take  to  the  outdoor  experimentation. 


! 

DisruptorUGV 


Sensor  UGV 
Real  world 


(b)  Fixed  simulator 


(c)  Mobile  simulator 


Figure  12  The  platform-in-the-loop  simulator 
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Safety,  E-Stop,  Freeze  &  Lost-link 

To  reduce  the  risk  of  injury  caused  by  the  UGVs,  a  number  of  safety  features  will  be  incorporated  in  the  UGVs. 
First,  each  UGV  is  equipped  a  termination  mechanism  (“E-stop”)  that  deactivates  all  the  ESCs  electrically  by 
disconnecting  their  power  and  applying  brake  on  all  the  motors  mechanically.  When  any  equipment  failure  or 
dangerous  behaviors,  such  as  out  of  boundary,  crashing  and  speeding,  are  observed  by  any  of  the  onboard 
sensors,  operators  or  judges,  the  MCU,  laptop,  operators  or  judges  can  activate  the  “E-stop”.  The  activation  by 
operators  is  achieved  through  the  HMI  in  the  BS  and  the  wireless  module  2  connected  to  the  laptop,  as  shown 
in  Figure  4.  On  the  other  hand,  the  judges  can  remotely  trigger  “E-stop”  via  handheld  remote  controls  and  a 
separate  wireless  module  connected  directly  to  the  MCU  and  E-stop,  regardless  the  status  of  the  laptop. 

Second,  an  administrative  stop  (“Freeze”)  is  implemented  on  each  UGV.  “Freeze”  is  achieved  by  the  MPU 
sending  commands  to  stop  all  ESCs  until  receiving  a  command  to  cancel  “Freeze”.  “Freeze”  can  be  triggered 
in  a  manner  similar  to  “E-stop”,  but  it  differs  from  “E-stop”  in  that  it  does  not  apply  mechanical  brake  on  and 
does  not  cut  power  to  each  motor.  Third,  when  each  UGV  starts,  the  laptop  and  MCU  check  and  ensure  all  the 
onboard  equipments  functioning  well  before  activating  the  ESCs.  Last,  in  the  event  of  lost-link  to  or  unstable 
communication  with  the  BS,  the  UGV  triggers  “Freeze”  immediately  as  the  operators  cannot  monitor  the  UGV 
status. 


Communications  Architecture 

To  improve  the  robustness  of  wireless  communications,  which  are  not  reliable  especially  in  non-line-of-sight 
conditions  or  out  of  range,  three  communication  protocols  are  utilized  to  suit  different  requirements  of 
reliability  for  remote  “E-stop”,  BS-UGV  and  UGV-UGV  communications.  For  remote  “E-stop”,  the  judges 
can  remotely  trigger  “E-stop”  through  their  handheld  devices,  which  send  “E-stop”  commands  continuously  to 
a  dangerous  UGV.  Once  the  onboard  MCU  receives  an  “E-stop”  signal  that  passes  a  checksum  test,  “E-stop” 
will  be  triggered.  Since  the  judges  who  observe  dangerous  behaviors  of  the  UGV  are  usually  very  close  to  the 
UGV,  the  wireless  communication  is  very  reliable. 

Besides,  the  BS-UGV  communication  uses  the  TCP/IP  protocol  that  is  very  reliable  due  to  its  abilities  of  error 
detection  and  retransmission  of  lost  data.  The  reliability  of  the  BS-UGV  communication  ensures  that  the 
operators  in  the  BS  can  monitor  all  UGVs’  faithful  sensor  readings  and  stop  some  UGVs  if  necessary  for 
safety  purposes.  Because  the  UGV-UGV  communication  does  not  require  high  reliability,  it  uses  multicast  IP, 
which  is  less  reliable,  for  high-speed  communication. 


Spectrum  Plan  &  Usage 

The  spectrum  usages  are  for  BS- 
UGV  and  UGV-UGV 

communications  and  remote  “E- 
stop”.  All  of  them  are  planned  to  use 
2.4  GHz  spectrum,  which  is  open 
and  safe  for  public.  They  can  share 
the  same  2.4  GHz  spectrum  by  using 
different  channels  without  interfering 
each  other.  To  reduce  interferences 
to  other  2.4  GHz  wireless  devices 
near  the  MAGIC  2010  field  and  to 
extend  the  range  of  UGV-BS 

communication,  directional  antennas 
pointing  to  the  field  are  used  in  the 
BS. 
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Figure  13  Gantt  chart 
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Test  Plan 


The  Virginia  Tech  team  possesses  a  number  of  UGVs  in  various  sizes  including  those  shown  in  Figure  6. 
Since  experimental  tests  can  be  performed  using  the  existing  UGVs  at  any  time,  tests  of  perception  will  start  in 
October,  2009  while  UGVs  for  MAGIC  2010  are  being  developed.  Figure  13  shows  a  Gantt  chart  indicating 
testing  period  as  well  as  development,  fine-tuning  and  simulation  periods.  All  the  developments  are  to  be 
completed  by  July,  2010,  and  the  performance  tests  of  integrated  UGV  system  will  start  subsequently  and 
continue  in  a  similar  test  site  offering  a  large  outdoor  space  and  indoor  buildings  until  the  system  is  sent  to 
Australia. 

Figure  14  shows  a  5-acre  JOint  Unmanned  Systems  Testing,  Experimentation  and  Research  (JOUSTER)  test 
site,  which  is  located  in  the  Danville  campus  of  Virginia  Tech.  The  6.6  million  dollar  JOUSTER  project 
sponsored  by  US  Department  of  Defense  has  created  one  of  the  best  university  test  environment  for  unmanned 
systems  research  with  various  research  facilities  and  unmanned  vehicles.  The  land  and  the  research  building 
will  be  used  as  the  test  environment  for  the  rehearsal  of  MAGIC  2010. 


(a)  JOUSTER  test  site 


(b)  View  from  cxxitrol  room 


Figure  14  JOUSTER  test  site 
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DEVELOPED  SYSTEM 


COOPERATIVE  CONTROL  STRATEGY 


Central  and  Decentralized  Control 

In  accordance  with  the  strategy 
proposed  in  Figure  1,  the  proposed 
strategy  has  incorporated  two  types  of 
hierarchical  control  approaches.  Figure 
15  shows  the  highest  control  approach, 
which  is  the  discrete  decision  control  to 
be  centrally  made  by  the  lead  UGV.  In 
order  to  enable  robot  control  as 
decentrally  as  possible,  the  lead  UGV 
makes  a  discrete  decision  whenever 
necessary.  Such  circumstances  include 
the  vehicle  allocation  during  mapping 
and  vehicle  allocation  after  001 
detection.  In  the  vehicle  allocation 
during  mapping,  the  lead  UGV  allocates 
an  area  to  each  UGV  so  that  each  UGV 
can  be  decentrally  controlled.  This  also 
enables  global  mapping  at  maximum 
efficiency  because  of  the  introduction  of 
no  overlapping  area.  Vehicle  allocation 
after  001  detection,  meanwhile, 
allocates  a  disruptor  UGV  to  the 
detected  001.  The  allocated  UGV 
therefore  neutralizes  the  detected  001,  and 


Knowl  0cl^  6 

Decision 

base 

maker 

Figure  15  Central  decision  making  by  lead  UGV 

the  other  disruptor  UGVs  can  work  independently  for  their  own  tasks. 


Bayesian  STLAM 


Figure  16  shows  the  illustration  of  the 
Bayesian  search,  tracking  and  localization 
techniques  implemented  in  the  developed 
system.  Due  to  the  near-Gaussian  nature 
of  the  likelihood  of  observed  001, 
tracking  is  carried  out  using  Extended 
Kalman  Filter  (EKF).  Search,  on  the 
other  hand,  is  performed  by  the  element- 
based  method  because  the  likelihood  is 
heavily  non-Gaussian.  Similarly  to 
tracking,  localization  occurs  when  an 
001  is  within  the  field  of  view  (FoV). 
Therefore,  EKF  is  used  for  localization. 

Tracking  and  localization  are 
computationally  inexpensive  processes 
since  only  the  mean  and  covariance  need 
to  be  updated.  They  are  thus  calculated 
on  a  central  processing  unit  (CPU).  On 
the  other  hand,  search  requires  grid-by- 
grid  computation,  and  a  graphics 


Bayesian  search  [Bourgault, 

Furukawa  &  Durrant- Whyte,  2001] 

Bayesian  Search  and  Tracking  (SAT) 
[Furukawa  &  Durrant- Whyte,  2005] 

Element-based  method  for  search 
[Furukawa  &  Du  riant- Whyte,  2006] 

Element-based  method  using  GPU 
[Furukawa  &  Durrant- Whyte,  2008] 

Hybrid  EM/EKF  for  SAT  [Mak,  Tong  & 
Furukawa,  2010] 

Search  for  Indoor/Outdoor  OOls 
[Ryu  and  Furukawa,  2010] 

Simultaneous  Localization  and  Mapping 
(SLAM)  [Dissanayake  &  Durrant- Whyte, 
1997] 

Element-based  SLAM  [Ryu  and 
Furukawa,  2009] 


Figure  16  Search,  tracking  and  localization 
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processing  unit  (GPU)  is  additionally  utilized. 

Figure  17  illustrates  the  specifications  of  the 
developed  occupancy  mapping  technique  and 
one  of  the  map  results.  The  developed 
occupancy  element  mapping  (OEM) 
technique  differs  from  the  conventional 
occupancy  grid  mapping  (OCM)  technique 
only  in  cell  representation.  The  difference 
by  the  use  of  finite  elements  with  shape 
function  enables  the  OEM  to  be  more 
accurate  than  the  OCM  without  much 
increase  in  computation  time. 

The  OEM  is  implemented  to  be  carried  out 
on  a  nodal  basis.  The  OEM  has  therefore 
been  implemented  on  a  GPU  so  that  the 
computational  efficiency  can  be  dramatically 
improved. 

Figure  17  Mapping 

PLATFORM-  AND  HARDWARE-IN-THE-LOOP  SIMULATOR  AND  PERFORMANCE 
EVALUATION 

Figure  18  shows  the  design  of  the  platform-  and  hardware-in-the-loop  simulator  (PHILS),  which  enables  the 
simulation  and  performance  evaluation  of  the  multi-UGV  system.  In  order  for  simulation  and  performance 
evaluation  in  a  real-time  environment,  the  system  consists  of  multiple  computers,  multiple  monitors  and  a  Gigabit 
switch.  The  visualizer  that  creates  a  virtual  environment  is  FlightGear,  which  is  an  open-source  tool  originally 
developed  for  flight  simulation.  In  the  creation  of  a  virtual  environment  with  a  system  having  multiple  computers,  a 
FlightGear  server  is  run  on  a  computer  having  Linux  as  an  operating  system.  Computers  as  many  as  UGVs,  on  the 
other  hand,  each  runs  a  FlightGear  client  and  share  the  environment  produced  by  the  FlightGear  server.  These 
computers  are  also  each  used  to  calculate  the  motion  of  a  UGV.  It  is  often  useful  to  visually  monitor  the  motion  and 

sensing  of  each  UGV.  For 
this  reason,  a  monitor  is 
connected  to  each 
computer.  Each  computer 
has  a  GPU,  so  graphics  can 
be  shown  at  accelerated 
speed.  Since  FlightGear 
allows  several  views  from 
a  vehicle,  these  views  can 
be  shown  on  each  monitor. 
Additionally,  the  field  of 
view  of  a  camera  mounted 
on  a  UGV  has  been  also 
made  possible  to  monitor. 
Motion  of  OOIs  need  to  be 
also  updated  in  real  time, 
but  it  is  not  important  to 
monitor  their  motion.  In 
order  for  real-time 
calculation,  the  multiple 
OOIs  have  been 

.  «  .  n  o  .  .  implemented  on  a  GPU. 

Figure  18  Design  of  platform-  and  hardware-in-the-loop  simulator  elements  consisting  a 


Occupancy  element  mapping  [Ryu  and  Furukawa,  2009] 

0:  Unoccupied 
0.5;  Unexplored 
1 :  Occupied 

Cell  representation:  Element  with  shape  function 
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real-time  virtual  environment  in  a  multi-computer  environment,  such  as  terrain  maps,  time  and  communication  are 
managed  by  an  environment  server.  Computers  are  connected  via  a  Gigabit  switch.  This  is  primarily  to  achieve 
real-time  simulation  but  also  to  control  wireless  communication  environments,  which  generally  has  the 
communication  speed  of  the  order  of  lOOMbs  or  less.  In  addition  to  communication  delay,  it  is  also  possible  to 
introduce  communication  loss.  The  communication  delay  and  loss  are  controlled  by  the  environment  server. 


Shown  at  the  bottom  half  are  the  computers  that  are  to  be  tested  in  real  environments.  These  include  computers  to 
be  mounted  on  UGVs  and  a  Base  Station  computer.  In  order  to  monitor  the  performance  of  capability  of  autonomy, 
the  PHILS  provides  a  monitor  for  each  UGV. 


Figure  19  shows  the  developed  PHILS.  A  big  screen  is  connected  to  one  of  the  computers  with  FlightGear  client 
and  shows  the  motion  of  all  the  UGVs.  Computers  calculating  UGV  motions  are  each  connected  to  a  40  inch 
monitor.  Base  Station  computer  and  on-board  computers  are  located  on  the  right-hand  side.  The  19  inch  monitors 
connected  to  the  on-board  computers  each  show  a  map  created  and  001  locations  identified  by  a  UGV,  whereas  the 
monitor  of  the  Base  Station  shows  a  combined  map. 


Computers  running  vehicle  dyiodmics 


Aerial  view  Computers  cr^a^r^  i*FlightGea 


Ground  control  station  Onboard  computers 


environment 


Figure  19  Platform-  and  hardware-in-the-loop  simulator 


UNMANNED  GROUND  VEHICLES 

Figure  20  shows  three  UGVs  developed  during  the  project.  In  order  to  identify  the  best -performance  UGV,  three 
UGVs  have  different  designs  and  configurations.  UGV  1  utilizes  Ackerman  steering  drive  train  and  thus  is  good  at 
high-speed  drive.  Field  tests  have  shown  that  the  maximum  speed  is  over  20  km/h.  On  the  other  hand,  the 
Ackerman  steering  driven  train  does  not  have  high  maneuverability.  It  has  been  generally  developed  to  drive  in 
outdoor  environments,  which  generally  have  less  obstacles  and  thus  enable  high-speed  drive.  Sensors  mounted  on 
the  UGV  1  include  GPS  for  outdoor  navigation,  LiDAR  for  obstacle  avoidance  and  mapping  and  camera  for  001 
detection  and  localization.  UGVs  2  and  3,  meanwhile,  have  a  differential  steering  platform.  The  UGVs  therefore 
achieve  high  maneuverability  at  the  expense  of  high  velocity.  UGV  2  has  IMU,  compass  and  encoders  in  addition  to 
the  GPS,  LiDAR  and  camera.  The  compass  and  IMU  makes  the  UGV  localization  in  indoor  environments,  and  the 
encoders  further  enables  the  indoor  localization  even  when  no  landmarks  can  be  observed.  The  UGV  however  has 
only  three  wheels  and  thus  does  not  drive  fast.  UGV  3  has  all  the  sensors  of  UGV  2  except  encoders.  As  a  result,  it 
is  essential  that  UGV  3  see  some  landmarks  for  successful  continuous  localization  unlike  UGV  2.  However,  UGV  3 
can  run  as  fast  as  15  km/h  unlike  UGV  2,  the  maximum  speed  of  which  is  around  6  km/h. 
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UGV  3 


Figure  20  Developed  UGVs 


TEST  AREA  CONSTRUCTION 

Figure  21  shows  the  test  site 
arranged  for  Team  VaCAS.  The 
test  site  is  part  of  Computational 
Multiphysics  Systems 

Laboratory  in  Institute  for 
Advanced  Learning  and 
Research,  Danville,  VA,  which 
is  directed  by  the  principal 
investigator.  The  test  site 
utilizes  two  buildings  and  one 
outdoor  field.  The  Ground 
Control  Station  (GCS)  is  located 
in  one  building  whereas  another 
building  provides  an  indoor 
environment.  The  GCS  is  about 
100  m  away  from  the  outdoor 
field  and  the  indoor  test  area  to 
satisfy  the  requirements. 

Figure  22  shows  the  dimensions 
of  the  outdoor  indoor  test  areas. 

Two  empty  rooms  were  utilized 
to  set  up  indoor  obstacles.  The 
obstacles  were  all  made  to 
follow  the  guidelines  of  the 
MAGIC  documents  and  the  instructions  of  the  National  Institute  of  Standards  and  Technology  (NIST). 

Figure  23  and  Figure  24  show  the  outdoor  field  and  the  indoor  test  area  with  obstacles  constructed  according  to  the 
guidelines.  A  total  of  ten  boxes,  each  with  dimensions  of  4  m  by  4  m  by  1 .2  m,  have  been  constructed  and  placed  as 
instructed.  Other  obstacles  include  fences. 


Figure  21  Test  site 
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Figure  22  Dimensions  of  outdoor  and  indoor  test  areas 


2  more  boxes  being  built 


Mobile  001  Area 
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Figure  23  Outdoor  test  area 

Figure  25  shows  the  capability  of  monitoring  the  test  areas  by  cameras.  A  total  of  seven  cameras  have  been 
installed,  and  the  installation  has  resulted  in  covering  90%  of  the  outdoor  and  indoor  test  areas  for  monitoring. 
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Simulation  and  Experimental  Results 

Figure  26  shows  two  virtual  outdoor  and  indoor  environments  that  were  modeled.  The  left  figure  is  the  test  area 
designed  by  following  the  guidelines  for  June  site  demonstration.  By  being  informed  of  the  coarse  configurations  of 
the  Phases  I-III  regions,  the  right  one  shows  a  test  area  resembling  the  demonstration  site  of  the  MAGIC  Final.  The 
developed  PHILS  can  easily  model  such  areas  and  allow  the  simulation  and  performance  evaluation  of  a  multi -UGV 
system. 


Figure  26  Virtual  outdoor  and  indoor  environments 


In  order  to  evaluate  its  performance  for  June  site  demonstration,  the  multi -UGV  system  was  used  to  map  the  outdoor 
and  indoor  test  areas  of  the  June  site  demonstration  and  search  for,  track,  neutralize  and  localize  static  and  mobile 
OOIs  using  PHILS.  The  number  of  UGVs  used  was  eight  as  proposed,  and  five  mobile  OOIs  and  two  static  OOIs 
were  placed  for  search,  tracking,  neutralization  and  localization. 

Figure  27  shows  the  resulting  map  and  the  locations  of  the  neutralized  mobile  and  static  OOIs  as  well  as  the 
locations  of  the  UGVs.  As  the  result  indicates,  nearly  the  entire  map  of  both  the  indoor  and  outdoor  test  areas  was 
created.  In  addition,  all  the  mobile  OOIs  and  static  OOIs  were  also  identified,  neutralized  and  localized.  The  result 
has  shown  the  capability  of  the  developed  multi -UGV  system  to  the  complete  the  mission  of  June  site  demonstration 
successfully. 


Figure  27  Results  of  mapping  and  localization  in  June  site  demonstration  test  area 
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Having  observed  the  ability  of  the  developed  multi -UGV  system  in  June  site  demonstration  test  area,  Figure  28 
shows  results  of  mapping  and  neutralization  in  a  MAGIC  Final  like  demonstration  area.  This  is  aimed  at 
investigating  not  only  the  capability  of  the  developed  multi-UGV  system  in  various  test  areas  and  conditions  but 
also  its  capability  to  win  in  the  competition.  As  evidently  seen,  an  accurate  map  covering  the  entire  Phases  I-III 
areas  has  been  created.  In  addition,  all  the  eight  mobile  OOIs  and  seven  static  OOIs  have  been  neutralized.  The 
results  demonstrate  the  capability  of  the  multi-UGV  system. 


Figure  28  Results  of  mapping  and  neutralization  in  MAGIC  Final  like  demonstration  area 

In  order  to  finally  demonstrate  that  the  virtual  test  results  are  comparable  to  the  real  test  results,  the  performance  of 
the  multi-UGV  system  was  investigated  for  the  mapping  of  indoor  rooms  in  both  virtual  and  real  environments. 
Figure  29  shows  the  resulting  map.  Since  details  of  some  corners  were  not  modeled  in  a  virtual  environment,  small 
difference  in  maps  can  be  seen.  However,  significant  similarities  in  addition  the  success  of  mapping  of  the  entire 
test  area  can  be  seen  in  both  the  results.  This  indicates  that  the  reliability  of  PHIS  in  quantifying  the  performance  of 
the  multi-UGV  system  in  a  real  environment. 


Figure  29  Maps  created  in  virtual  environment  (left)  and  real  environment  (right) 
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To  demonstrate  the  extensive  applicability  of  the  multi -UGV  system  in  the  cooperation  of  real  UGVs  and  virtual 
UGVs  in  the  same  environment,  Figure  30  shows  the  demonstration  of  three  real  UGVs  and  five  virtual  UGVs  in  the 
June  site  demonstration  test  area.  Similarly  to  the  results  by  eight  simulated  UGVs,  the  three  real  and  five  virtual 
UGVs  could  build  the  entire  map  appropriately.  The  system  has  been  upgraded  to  be  able  to  observe  real  UGVs  by 
virtual  UGVs  by  placing  real  UGVs  at  the  locations  measured  by  GPS  and  compass.  However,  real  UGVs  are  still 
not  able  to  observe  virtual  UGVs. 


Figure  30  Cooperation  by  three  real  UGVs  and  five  virtual  UGVs 
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